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(3) Interactive volume rendering method and system. 
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(57) In order to reduce the amount of re-calcu- 
lations needed when mapping functions to a 
color/opacity volume are changed, the proces- 
sing of a volume rendering method is separated 
into two portions, the first depending on the 
volume data (i.e., a portion determined by the 
volume data) and the second depending on the 
mapping to the color and opacity. Pre-proces- 
sing calculations will then be limited to the 
portion depending on the volume data, so as to 
thereby allow mapping functions to the color 
and opacity to be changed interactively. To this 
end, a "VIS volume" that consists of a data 
value, a. shaded intensity, and a surface 
strength, and is expressed by 3 bytes per one 
voxel, is calculated and ray-traced. The color 
and opacity at each sampling point are deter- 
mined when the VIS volume is sampled. Theref- 
ore, even if the mapping functions to the color 
and opacity are changed, it is not necessary to 
calculate the VIS module again. 
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Field of the invention 

The present invention relates to a method and system for interactive volume rendering. 
5 Prior art 

!n recent years ; volume rendering methods have come to be used for ima^e processing of GT scans, vis- 
ualization of the computed results of fluid analysis, visualization of measurement data of resource exploration, 
etc. The method by Levoy is known as the typical volume rendering method (M. Levoy, "Display of Surfaces 
10 from Volume Data," IEEE CG & A, Vol. 8, No. 3, pp. 29-37, 1988). 

In the following, Levoy's method will be described with reference to Figure 2 for a case in which volume 
data is assumed to be three-dimensional array data and a ray tracing method is employed as the volume data 
projection method. 

A volume value (block 21 2) at a lattice point (voxel) (i j,k) of volume data 200 is written as 
15 data(i,j,k). (1) 

A shaded color C(i,j,k) (block 21 0) and an opacity a(i,j,k) (block 214) at the lattice point (i,j,k) of a color/opa- 
city volume are calculated according to the following procedure. 

Initial color 202: Co(i,j,k) = {Ro(i ,j,k), G 0 (i j.k), B 0 (i ,j,k) (2) 
Ro(i,j,k) = ColorLUT(data(i j,k), 0) 
20 G 0 (»,j,k) = ColorLUT(data(i,j,k), 1) 

B 0 (i j,k) = ColorLUT(data(i,j,k), 2) 
In the above equations, "LUT" means table look-up operation. 

Initial opacity 208: ao(i,j,k) (3) 
oto(ij,k) = OpacityLUT(data(ij,k)) 
25 Gradient vector 204: Gv(i,j,k) (4) 

Gv(i j,k) = {Gvx(i,j,k), Gvy(i,j,k), Gvz(i j,k)} 
Gvx(i,j,k) = data(i+ 1,j,k) - data(i - 1,j,k) 
Gvy(ij,k) = data(i,j + 1,k) - data(i,j - 1,k) 
Gvz(ij,k) = data(i,j,k + 1) - data(i,j,k - 1) 
30 Magnitude 206 of gradient vector Gm(i,j,k) (5) 

Gm(ij.k) = lGv(i.j,k)| 
Shaded intensity l(i,j,k) (6) 
l(i,j,k) = ShadingLUT(Gv(i,j,k)) 
The shading intensity l(i,j,k) is omitted in Figure 2, but is used in calculating the shaded color 210 as de- 
35 scribed below. 

Surface strength S(i,j,k) (7) 
S(i,j,k) = StrengthLUT(Gm(i,j,k)) 
The surface strength S(i,j,k) is also omitted in Figure 2, but is used in calculating the opacity 214. Using 
the above parameters, the following calculations are performed. 
40 Shaded color 210: C(i,j,k) = {R(i,j,k), G(i,j,k), B(ij.k)} (8) 

C(i,j,k) = C 0 (i,j,k) * l(i,j,k) 
R(ij,k) = Ro(i,j,k) * l(i,j,k) 
G(i j.k) = G 0 (i j.k) * 
B(i,j,k) = B 0 (i,j,k) * l(i j,k) 
45 Opacity 214: a(i,j,k) (9) 

a(i,j,k) = ao(i,j,k) * S(i,j,k) 

The color/opacity volume consists of the shaded color (three components of R, G and B) and the opacity. 
More specifically, for the lattice point (i,j,k), the volume has the four components of 

R(i,j,k), G(i,j,k), B(i,j,k) and a(i,j,k). (1 0) 
50 In addition to the above components, the original volume data 212 needs to be used. Therefore, if each 

of the five components is expressed by 1 byte (a range of 256 levels, 0-255), 5 bytes are required for one lattice 
point (voxel) in Levoy's method. 

In Levoy's method, the color/opacity volume is ray-traced to calculate a sample color 218, a sample data 
value 220, and a sample opacity 222, thereby generating an image. However, if the color and opacity of the 
55 . image is changed interactively, the mapping functions of the color and opacity are changed. Those mapping 
functions correspond to above-described 

ColorLUTO and OpacityLuTO. (11) 
Therefore, the color/opacity volume needs to be calculated again every time the mapping functions of the 
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color and opacity are changed. Furthermore, since there exists the calculation processing stage using the map- 
ping functions of the color and opacity before the ray tracing (sampling), not only the sampling calculation but 
also the color/opacity volume calculation itself needs to be performed again if the mapping function of the col- 
or/opacity volume is changed. Therefore, for example, even in the case of high-opacity data, that is, even where 
5 data of only voxels close to a viewpoint contribute to a generated image, all the voxels need to be subjected 
to re-calculations. That is, extra calculation time is consumed to perform wasteful calculations. 

Other than Levoy's method, there exists a prior art technique disclosed in GB-A-2 231 759. This technique 
relates to a method and system for displaying a volume object on a computer graphics display device, and con- 
sists of the steps of storing in a memory volume data that is represented in a volume space and generating a 
10 mapping function for mapping each unit of volume data to coordinates of a geometrical graphics element 

However, the above publication has no disclosure as to re-mapping of the volume data. 

SUMMARY OF THE INVENTION 

15 An object of the present invention is to reduce the amount of recalculation needed when mapping functions 

to a color/opacity volume are changed in a volume rendering method. 

Another object of the invention is to avoid an artifact (moire pattern) that would occur in a generated image 
in association with interpolation of the color and opacity. 

The basic concept of the present invention resides in that processing of a volume rendering method is sepa- 
20 rated into a portion which depends on volume data (i.e., a portion determined by the volume data) and a portion 
which depends on mapping to the color and opacity and only the portion depending on the volume data is cal- 
culated as pre-processing, thereby allowing mapping functions to the color and opacity to be changed inter- 
actively. To this end, a 'VIS volume" that consists of a data value, a shaded intensity, and a surface strength 
and is expressed by 3 bytes per voxel is calculated and ray-traced. The color and opacity at each sampling 
25 point are determined when the VIS volume is sampled. Therefore, even if the mapping functions to the color 
and opacity are changed, it is not necessary to calculate the VIS volume again. Since interpolation is performed 
for each sampling point using the original data values, no artifact (moire pattern) occurs in a generated image 
in association with the interpolation of the color and opacity. 

According to the invention, since no re-calculations are required in the pre-processing portion when the 
30 mapping functions to the color and opacity are changed, the re-calculation time in association with a change 
of the mapping functions to the color and opacity is reduced to 10-80%. Furthermore, the calculation time of 
the pre-processing portion is reduced to about 80%, the image generating time is kept the same or reduced 
to some extent, and the necessary memory space is reduced to about 60%. 

As described above, according to the Invention, since the VIS volume need not be re-calculated even 
35 though the mapping functions to the color and opacity are changed, the image re-calculation and re-display 
can be performed quickly in response to the change of the color and opacity. 

While conventionally 5 bytes are needed for one voxel, only 3 bytes are needed in the invention. This will 
contributes to reduction of the necessary memory space. 

Furthermore, since interpolation for each sampling point is performed using original data values, artifact 
40 (moire pattern) occurrence in a generated image in association with the interpolation of the color and opacity 
can be improved. 

By combining the invention and a successive refinement type image generation method, it becomes pos- 
sible to change a generated image quickly and interactively in response to a change of mapping functions to 
the color and opacity in generating a three-dimensional image according to a volume rendering method, even 
45 on a general-purpose work station, i.e., without using special hardware. 

Brief Description of the Drawings : 

Figure 1 is a block diagram showing constitution of a system to practise the present invention; 
50 Figure 2 is a block diagram showing ray trace processing in a conventional technique; and 

Figure 3 is a block diagram showing ray trace processing in the invention 

Detailed Description of the Invention 

55 Figure 1 is a block diagram showing the hardware configuration presumed in the execution of the present 

invention. The system of Figure 1 has ah ordinary configuration including a keyboard 102 used for inputting 
data to effect interactive operation; a CPU 104 for performing calculations; a memory 108 for loading a system 
for data processing, temporarily storing calculation results of the CPU 1 04, and providing a cache for the CPU 
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104; a CRT 110; a frame buffer 106 for dynamically holding, in a rewritable manner, data to be displayed on 
the CRT 110; a magnetic disk 114 for permanently storing a system program, application programs and proc- 
essing data; and a bus 11 6 for connecting the above devices to enable data exchange therebetween. Although 
not shown in Figure 1, a pointing device such as a mouse is usually connected to the above system for the 

5 purpose of interactive operation, in addition to the keyboard 1 02. Processing modules and volume data shown 
in Figure 3 are normally stored in the magnetic disk 114, and loaded by the CPU 104 into the memory 108 via 
• the bus 116 at the start of processing. In response to an interactive operation by an operator through thp key- 
board 102 (or the mouse (not shown)), the CPU 104 performs calculations on the data of the memory 108 and 
forwards the calculation results to the frame buffer 106, thereby enabling a desired graphics display. 

10 With reference to Figure 3, a volume rendering technique of the invention will be described below. In Figure 

3, a data value 308 of a lattice point (voxel) (i,j,k) of volume data 300 or of the voxel closest to the lattice point 
(i j,k) is written as 

data(ij.k). (12) 

Various parameter values at the lattice point (i,j,k) of a VIS volume are calculated as follows. 
15 Gradient vector 302: Gv(ij,k) (13) 

Gv(i,j,k) = {Gvx(i j f k), Gvy(i ,j,k), Gvz(i,j,k)} 
Gvx(i,j,k) = data(i + 1j,k) - data(i - 1,j,k) 
Gvy(ij.k) = data(i,j+ 1,k) - data(ij - 1,k) 
Gvz(i,j,k) = data(ij,k+1) - data(i,j,k- 1) 
20 Magnitude 304 of gradient vector Gm(i,j,k) (14) 

Gm(i,j,k) = |Gv(ij,k)| 
Using the above values, the following calculations are made. 

Shaded intensity 306: l(ij,k) (15) 
l(i,j,k) = ShadingLUT(Gv(i,j,k)) 
25 Surface strength 310: S(i,j,k) (16) 

S(i,j,k) = StrengthLUT(Gm(i,j,k)) 
In the invention, the VIS volume 312 consists of the data value, shaded intensity, and surface strength, 
and the lattice point (i j,k) has the three components of 

data(ij.k), l(ij,k) and S(i,j,k). (17) 
30 If each competent is expressed by 1 byte (a range of 256 levels, 0-255), 3 bytes are required for one lattice 

point (voxel) in this method. 

According to the invention, an image is generated by ray-tracing the VIS volume. 
It is assumed that a data value 316, a shaded intensity 314, and a surface strength 31 8 at the nth sampling 
along a ray are expressed as 
35 data(n), l(n) and S(n). (18) 

A shaded color C(n) and ah opacity a(n) are calculated in the following manner. 

Initial color 320: C 0 = {Ro(n), Go(n), B 0 (n)} (19) 
Ro(n) = ColorLUT(data(n), 0) 
Go(n) = Colort_UT(data(n), 1) 
40 B 0 (n) = ColorLUT(data(n), 2) 

Initial opacity 322: ao(n) (20) 
ao= OpacityLUT(data(n)) 
Therefore, the following calculations are made. 

Shaded color 324: C(n) = {R(n), G(n), B(n)} (21) 
45 C(n) = C 0 (n) * l(n) 

R(n) = Ro(n) * l(n) 
G(n) = Go(n) * l(n) 
B(n) = B 0 (n) * l(n) 
Opacity 326: a(n) (22) 
so a(n) = cto(n) * S(n) 

Mapping functions of the color and opacity correspond to 

ColorLUT() and OpacityLuTO. (23) 
However, even if the mapping functions of the color and opacity are changed, they do not influence the 
VIS volume and, therefore, the VIS volume need not be calculated again. In other words, in the invention, since 
55 the mapping functions of ColorLUT() and OpacityLuTO are carried out in the ray tracing (sampling) stage, no 
re-calculations are required for the VIS volume even if the mapping functions of the color and opacity are 
changed. By contrast, in the conventional constitution of Figure 2, since the mapping functions of CoiorLUTO 
and OpacityLUTQ are carried out in the stage before the ray tracing stage, re-calculations for the color/opacity 
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volume should be performed every time the mapping functions of the color and opacity are changed. In the 
light of the facts that a considerable amount of calculation is usually required to re-calculate the color/opacity 
volume and that in interactive operation the color and opacity of an image are changed relatively frequently, 
it is understood that the invention provides a great advantage in terms of saving the amount of calculation com- 

5 pared to the conventional technique of Figure 2. 

Comparative experiments were performed for the following cases, in which the method according to this 
invention and Levoy's method were incorporated into an interactive volume rendering tool employing an image 
generation method based on a successive refinement technique developed by the present inventors. 
Case 1 : display of an epidermis based on X-CT data of 256 x 256 x 133 
10 Case 2: display of a skull based on X-CT data of 256 x 256 x 1 33 

Case 3: display of a translucent epidermis and a skull based on X-CT data of 256 x 256 x 1 33 
Case 4: display of a brain based on MRI data of 252 x 252 x 139 
Case 5: display of a color mapping image of PET data of 128 x 128 x 61 

15 " 

Method of Levey 1 s Ratio 
invention method 

Case 1 Pre-processing 123.68 148.74 0.83:1 

on t*- me (s) 

M Image generation 18.70 17.85 1.05:1 

time (s) 

Total calculation 142.38 166.59 0.85:1 

time (s) ^ 

Re-calculation 18.70 166.59 0.11:1 - 

time (s) 

6 Total number of 256,056 241,718 1.06 si 
sampling 
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Case 2 Pre-processing 
time (s) 

Image generation 
time (s) 

Total calculation 

time (r) 

Re-calculation 

time (s) 

Total number of 

sampling 

Case 3 Pre-processing 
time (s) 

Image generation 
time (s) 

Total calculation 

time (s) 

Re -calculation 

time (s) 

Total number of 

sampling 

Case 4 Pre-processing 
time (s) 

Image generation 
time (s) 

Total calculation 

t ime ( s ) 

Re-calculation 

time (s) 

Total number of 

sampling 

Case 5 Pre-processing 
time (s) 

Image generation 
time (s) 

Total calculation 

time (s) 

Re-calculation 

time (s) 

Total number of 

sampling 


31.81 

42.04 

0.76:1 

28.49 

26.70 

1.07:1 

60.30 

68.74 

0.88:1 

28.49 

68.74 

0.41:1 - 

288,743 

266,793 

1.08:1 

151 .99 

180.22 

0 . 84 : 1 

35.38 

50.11 

0.71:1 

187.37 

230.33 

0.81 :1 

35.38 

230.33 

0.15:1 - 

804 , 677 

1 ,396,892 

0.58:1 

54.93 

66.49 

0.83:1 

21.74 

20.56 

1.06:1 

76.67 

87 .05 

0.88:1 

21.74 

87.05 

0.25:1 - 

307,907 

307,258 

1.00:1 

3.36 

4.25 

0.79:1 

25.82 

27.14 

0.95:1 

29.18 

31.39 

0.93=1 

25.82 

31.39 

0.82:1 

513,370 

499,714 

1.03:1 


40 In the above tables, the "pre-processing time" and the "image generation time" respectively correspond 

to the times required for the VIS volume calculation and the ray tracing of the VIS volume in the method of the 
invention, and respectively correspond to the times required for the calculation of the color/opacity volume and 
the ray tracing of the color/opacity volume in Levoy's method. The "total calculation time" is the time required 
for the first generation of a three-dimensional image, and the "re-calculation time" is the time required for the 

45 generation of a three-dimensional image from the second time onward, where the mapping functions to the 
color and opacity are changed. According to the method of the invention, the re-calculation time includes only 
the image generation time, because re-calculations need not be performed in the pre-processing portion even 
when the mapping functions to the color and opacity are changed. 
In the above tables, the respective times are CPU times (sec). 

50 


Claims 

1. A computerized volume rendering method, comprising the steps of: 
55 (a) providing volume data; 

(b) constructing a voxel at each lattice point of the volume data by a data value, an intensity, and a 
surface strength; 

(c) determining, for each ray, a data value, an intensity, and a surface strength at each sampling point 
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along the ray, and calculating a sample color and a sample opacity based on the data value, intensity, 
and surface strength thus determined; and 

(d) determining, for each ray, the pixel value of an image to be generated by accumulating the sample 
color values and the sample opacities calculated in step (c) along the ray. 

A computerized volume rendering method, comprising the steps of: 

(a) providing volume data; 

(b) constructing a voxel at each lattice point of the volume data by a data value, an intensity, and a 
surface strength;. 

(c) determining, for each ray, the data value, the intensity, and the surface strength of the voxel closest 
to each sampling point along the ray, and calculating the sample color and sample opacity of the voxel 
closest to the sampling point based on the data value, intensity, and surface strength thus determined; 
and 

(d) determining, for each ray, the pixel value of an image to be generated by accumulating the sample 
color values and the sample opacities calculated in step (c) along the ray. 

A volume rendering method according to claim 1 or 2, wherein the data value, intensity, and surface 
strength of the voxel each consist of 1 byte, which means one voxel consists of 3 bytes. 

A computerized volume rendering method, comprising the steps of: 

(a) providing volume data; 

(b) constructing a voxel at each lattice point of the volume data by a data value, an intensity, and a 
surface strength; 

(c) determining an opacity mapping function and a color mapping function in response to designation 
of an opacity and a color from an operator; 

(d) determining, for each ray, a data value, an intensity, and a surface strength at each sampling point 
along the ray, and calculating a sample color and a sample opacity from the data value, intensity, and 
surface strength using the mapping functions determined in step (c); and 

<e) determining, for each ray, a pixel value of an image to be generated by accumulating the sample 
color values and the sample opacities calculated in step (d) along the ray. 

A volume rendering method according to claim 4, wherein the data value, intensity, and surface strength 
of the voxel each consist of 1 byte, which means one voxel consists of 3 bytes. 

A volume rendering method according to claim 4, wherein each of the mapping functions is defined by a 
look-up table. 

A volume rendering system comprising: 

(a) a means for storing volume data; 

(b) a means for sampling the volume data, constructing a voxel at each of the sampling points by a 
data value, an intensity, and a surface strength, and storing the voxel thus constructed; 

(c) a means for determining an opacity mapping function and a color mapping function in response to 
designation of an opacity and a color from an operator; 

(d) a means for determining, for each ray, a data value, an intensity, and a surface strength at each 
sampling point along the ray, and calculating a sample color and a sample opacity from the data value, 
intensity, and surface strength using the mapping functions determined by means (c); and 

(e) means for determining, for each ray, the pixel value of an image to be generated by accumulating 
the sample color values and the sample opacities calculated by means (d) along the ray. 

A volume rendering system according to claim 7, wherein the data value, intensity, and surface strength 
of the voxel each consist of 1 byte, which means one voxel consists of 3 bytes. 

A volume rendering system according to claim 7, wherein each of the mapping functions is defined by a 
look-up table. 
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